{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j4qMNV_533ls"
      },
      "source": [
        "##### Copyright 2025 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "urf-mQKk348O"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iS5fT77w4ZNj"
      },
      "source": [
        "# Gemma - Run with Ollama Python library\n",
        "\n",
        "Author: Onuralp Sezer\n",
        "\n",
        "*   GitHub: [github.com/onuralpszr](https://github.com/onuralpszr/)\n",
        "*   X: [@onuralpszr](https://x.com/onuralpszr)\n",
        "\n",
        "Description: This notebook demonstrates how you can run inference on a Gemma3 model using  [Ollama Python library](https://github.com/ollama/ollama-python). The Ollama Python library provides the easiest way to integrate Python 3.8+ projects with Ollama.\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_3]Using_with_Ollama_Python_Inference_with_Images.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FF6vOV_74aqj"
      },
      "source": [
        "## Setup\n",
        "\n",
        "### Select the Colab runtime\n",
        "To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma model. In this case, you can use a T4 GPU:\n",
        "\n",
        "1. In the upper-right of the Colab window, select **▾ (Additional connection options)**.\n",
        "2. Select **Change runtime type**.\n",
        "3. Under **Hardware accelerator**, select **T4 GPU**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4tlnekw44gaq"
      },
      "source": [
        "## Installation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AL7futP_4laS"
      },
      "source": [
        "Install Ollama through the offical installation script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MuV7cWtcAoSV"
      },
      "outputs": [],
      "source": [
        "!sudo apt-get install pciutils\n",
        "!curl -fsSL https://ollama.com/install.sh | sh"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FpV183Rv6-1P"
      },
      "source": [
        "Install Ollama Python library through the official Python client for Ollama."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "0Mrj29SH-3OD"
      },
      "outputs": [],
      "source": [
        "!pip install -q ollama"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iNxJFvGIe48_"
      },
      "source": [
        "## Start Ollama\n",
        "\n",
        "Start Ollama in background using nohup."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "5CX39xKVe9UN"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "nohup: redirecting stderr to stdout\n"
          ]
        }
      ],
      "source": [
        "!nohup ollama serve > ollama.log &"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6YfDqlyo46Rp"
      },
      "source": [
        "## Prerequisites\n",
        "\n",
        "*   Ollama should be installed and running. (This was already completed in previous steps.)\n",
        "*   Pull the gemma3 model to use with the library: `ollama pull gemma3:4b`\n",
        "    *  See [Ollama.com](https://ollama.com/) for more information on the models available."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "oPU5dA1-B5Fn"
      },
      "outputs": [],
      "source": [
        "import ollama"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "NE1AWlucBza_"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "ProgressResponse(status='success', completed=None, total=None, digest=None)"
            ]
          },
          "execution_count": 34,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ollama.pull('gemma3:4b')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KL5Kc6HaKjmF"
      },
      "source": [
        "## Inference with Text 📝 and Image 🖼️\n",
        "\n",
        "Run inference using Ollama Python library."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-CjTl1ajarEE"
      },
      "outputs": [],
      "source": [
        "!curl -A \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36\" -o pexels-adria-masi-461420600-27372369.jpg \"https://images.pexels.com/photos/27372369/pexels-photo-27372369.jpeg?cs=srgb&dl=pexels-adria-masi-461420600-27372369.jpg&fm=jpg&w=1280&h=856\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MclgY4tLbpch"
      },
      "source": [
        "### Image Preview (Optional) 🖼️"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2mQbRM3JbZt2"
      },
      "outputs": [],
      "source": [
        "import cv2\n",
        "from google.colab.patches import cv2_imshow\n",
        "cv2_imshow(cv2.imread('pexels-adria-masi-461420600-27372369.jpg'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HN0nrhpmFUUB"
      },
      "source": [
        "### Generate"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VvBB_XoObyx1"
      },
      "outputs": [],
      "source": [
        "import ollama\n",
        "\n",
        "res = ollama.chat(\n",
        "\tmodel=\"gemma3:4b\",\n",
        "\tmessages=[\n",
        "\t\t{\n",
        "\t\t\t'role': 'user',\n",
        "\t\t\t'content': 'Describe this image:',\n",
        "\t\t\t'images': ['./pexels-adria-masi-461420600-27372369.jpg']\n",
        "\t\t}\n",
        "\t]\n",
        ")\n",
        "\n",
        "print(res['message']['content'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Vh_9LD_ocl6t"
      },
      "outputs": [],
      "source": [
        "!wget https://ollama.com/public/blog/wordart.jpg"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "utyVRlIYFvdm"
      },
      "source": [
        "#### Async client with OCR Usage 📝\n",
        "\n",
        "To make asynchronous requests, use the `AsyncClient` class."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JVvKQE0XF2kh"
      },
      "outputs": [],
      "source": [
        "import asyncio\n",
        "import nest_asyncio\n",
        "from ollama import AsyncClient\n",
        "\n",
        "nest_asyncio.apply()\n",
        "\n",
        "\n",
        "async def generate():\n",
        "    \"\"\"\n",
        "    Asynchronously generates a response to a given prompt using the AsyncClient.\n",
        "\n",
        "    This function creates an instance of AsyncClient and sends a request to generate\n",
        "    a response for the specified prompt. The response is then printed.\n",
        "    \"\"\"\n",
        "    # Create an instance of the AsyncClient\n",
        "    client = AsyncClient()\n",
        "\n",
        "    # Send a request to generate a response to the prompt\n",
        "    message={\n",
        "    'role': 'user',\n",
        "    'content': 'What does the text say ?',\n",
        "    'images': ['./wordart.jpg']}\n",
        "\n",
        "\n",
        "    async for part in await AsyncClient().chat(model='gemma3:4b', messages=[message], stream=True):\n",
        "      print(part['message']['content'], end='', flush=True)\n",
        "\n",
        "# Run the generate function\n",
        "asyncio.run(generate())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9V2TY2KQesS1"
      },
      "outputs": [],
      "source": [
        "!curl -A \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36\" -o pexels-pixabay-60582.jpg \"https://images.pexels.com/photos/60582/newton-s-cradle-balls-sphere-action-60582.jpeg?cs=srgb&dl=pexels-pixabay-60582.jpg&fm=jpg&w=1920&h=1346\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WFyi_zzWAwe7"
      },
      "source": [
        "### Object Counting 🧾"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UN20MoSv_S76"
      },
      "outputs": [],
      "source": [
        "from ollama import chat\n",
        "\n",
        "# Start a conversation with the model\n",
        "messages = [\n",
        "    {\n",
        "        \"role\": \"user\",\n",
        "        \"content\": \"How many Balls on the cradle?\",\n",
        "        \"images\": [\"./pexels-pixabay-60582.jpg\"]\n",
        "    },\n",
        "]\n",
        "\n",
        "# Get the model's response to the message\n",
        "response = chat(\"gemma3:4b\", messages=messages)\n",
        "print(response[\"message\"][\"content\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDr8VzvGIXFC"
      },
      "source": [
        "## Conclusion 🏆\n",
        "\n",
        "Congratulations! You have successfully run inference on a Gemma3 model using the Ollama Python library with VLM capabilities. You can now integrate this into your Python projects."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "[Gemma_3]Using_with_Ollama_Python_Inference_with_Images.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
